Skip to content

feat: enable mypy session for ndb#16691

Open
chalmerlowe wants to merge 9 commits intomainfrom
feat-enable-mypy-ndb
Open

feat: enable mypy session for ndb#16691
chalmerlowe wants to merge 9 commits intomainfrom
feat-enable-mypy-ndb

Conversation

@chalmerlowe
Copy link
Copy Markdown
Contributor

This PR enables the mypy session in noxfile.py for ndb and aligns it with the GAPIC generator template. Currently, it fails with 196 errors. This is pushed to allow farming out the work to resolve these errors or for separate review.

@chalmerlowe chalmerlowe requested a review from a team as a code owner April 16, 2026 14:42
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enables the mypy type checker in the noxfile.py for the google-cloud-ndb package, which was previously skipped. The feedback suggests improving the mypy command to correctly handle positional arguments and adding the --namespace-packages flag to ensure proper module resolution within the google namespace.

Comment thread packages/google-cloud-ndb/noxfile.py
@chalmerlowe chalmerlowe self-assigned this Apr 16, 2026
yield self._next_batch() # First time

assert self._batch is not None
assert self._index is not None
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to have assert as part of this code path? Usually this is only used in tests

Copy link
Copy Markdown
Contributor Author

@chalmerlowe chalmerlowe Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked for ways to provide type narrowing for mypy and assert is one way it can be done.

It is admittedly controversial for some in production code.

I went ahead and switched the asserts into if clauses throughout.

Comment thread packages/google-cloud-ndb/google/cloud/ndb/context.py
if isinstance(value, str):
try:
time_tuple = time.strptime(value, "%H:%M:%S")
st = time.strptime(value, "%H:%M:%S")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace st with a more meaningful name

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gemini suggested this refactor to clean up this code and reduce the number of if statements

import datetime
import time

def _time_function(values):
    # 1. Normalize input into a tuple of integers
    if len(values) == 1:
        val = values[0]
        if isinstance(val, str):
            try:
                st = time.strptime(val, "%H:%M:%S")
                t_tuple = (st.tm_hour, st.tm_min, st.tm_sec)
            except ValueError as e:
                _raise_cast_error(f"Format error: {e}, {values}")
        elif isinstance(val, int):
            t_tuple = (val,)
        else:
            _raise_cast_error(f"Invalid type {type(val)} for time()")
    elif 1 < len(values) < 4:
        t_tuple = tuple(values)
    else:
        _raise_cast_error(f"Invalid number of arguments: {len(values)}")

    # 2. Convert tuple to datetime.time using unpacking
    try:
        return datetime.time(*t_tuple)
    except (ValueError, TypeError) as e:
        _raise_cast_error(f"Time conversion error: {e}, {values}")

Copy link
Copy Markdown
Contributor Author

@chalmerlowe chalmerlowe Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parthea

These two code blocks are fundamentally the same with the exception of item # 2. Convert tuple to datetime.time using unpacking so my comments focus on that.

In your comment we assert that Gemini recommends this refactor to reduce the number of if statements.

Funnily enough:

Those if statements were also recommended by Gemini to enable greater type safety with mypy, because the datetime.time(*t_tuple) expression caused mypy errors.

google/cloud/ndb/_gql.py:793: error: Too many arguments for "time"  [call-arg]
google/cloud/ndb/_gql.py:793: error: Too many positional arguments for "time"  [misc]

Here is Gemini's rationale for the approach taken in this PR (this was the third change we made and Gemini felt it was the most important):

< BEGINNING of GEMINI's exposition >

  1. Exploding the "Splat" Operator (*t_tuple)

This is the most important change for Mypy:

Old: return datetime.time(*time_tuple)
The Problem: The * (splat) operator tells Python to unpack the tuple into arguments. However, datetime.time() accepts specific arguments: hour, minute, second, microsecond.
If time_tuple has a length of 1, it’s just the hour. If it's length 3, it's H, M, S.

Mypy cannot guarantee at compile-time that the length of the tuple matches what the function expects. It sees *time_tuple and worries you might be passing 10 arguments to a function that takes 4.

New:

if len(t_tuple) == 1:
    return datetime.time(t_tuple[0])
elif len(t_tuple) == 2:
    return datetime.time(t_tuple[0], t_tuple[1])
# ...

The Benefit: This is called Exhaustive Checking or Manual Unrolling.
By checking the len() explicitly, you are "narrowing" the type. Inside the if len(t_tuple) == 1 block, Mypy knows for a fact that t_tuple[0] exists and is the only argument.
This removes all ambiguity. Mypy can now prove the code is safe without needing a # type: ignore.

< END of GEMINI's exposition >

I am happy to resolve this in whatever way feels best to you, but I suspect that if I revert back to using the splat operator, we will have to go through another cycle of trying to find a way to keep mypy happy OR add one or more ignore pragmas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants